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Inferring parameters in science experiments 


Please think about this question (Exercise 3.3, page 47) 


A source emits particles; they decay at locations 
exponentially distributed with lengthscale A. De- 


says can be observed only if the location falls in a 
window (Тан: Emax) = (a,b) = (1,20). 

N авс: 
What is A? 


are observed at locations {21,...,: EN). 


Inferring parameters in science experiments 


Q: How would you have solved this problem before you heard about 
likelihood functions and Bayes' theorem? 


A source emits particles; they decay at locations 
exponentially distributed with lengthscale A. De- 
cays can be observed only if the location falls in a 
window (min; Cmax) = (a,b) = (1,20). 


N decays are observed at locations {21,...,: rN}. 


ч ум in 


Put data in bins, make histogram 


Put data in bins, make histogram, take log 


then ‘fit a straight line" 
(how do we handle 
zero counts?) 
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Other ideas 


@Chi-squared 
minimize with respect to lambda 
also depends on bins 


(other measure of ‘error’ between theory and data 
@Find how the data-mean is expected to vary with lambda 


Other 'estimators', depending on other data-properties 


The answer using Bayes's theorem 
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Or, making explicit the underlying assumptions - 
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The answer using Bayes's theorem 


Buffers Files Tools Edit Search Mule Help 


gnuplot commands to illustrate how simple Bayesian inference is 


Define the density P(xI Lambda) 
STAN 
205 = exp -а/1) - есь) 
Si 

22 


In The First plot illustrates the predictive 
| distributions for different values of lambda 
plot (1:203(0:0,253 pix.2) + EN 
PCS) є t 555 
гало t 


| Nou imagine we receive data (x ) = (3, 5, 12) 
| This Le the definition of the likelihood Function 
liked) = PED * pé" ваг» 

Js The next plot shows the thr 


Is in the likelihood function 
set logscale x 


factors, Pix ni lambda). 


Set xlabel 

Plot C0,3:140) p(3,x) t E 
HOPES ES 
раю t 

la Finally, plot the likelihood 

plot (0,3:140) likeGO t wıs 


ас: /home/mackay/1tp/octave/decay/gnudb 


(ве £ 
ai 5-20 


The answer using Bayes's theorem 


Search Mule Help 


lambda=5 


Pla |lambda=s に 
P(x|lambda=10) 一 


The answer using Bayes's theorem 


P(x|lambda=2) 
P(xllambda=5) 一 一 一 
P(xllambda=10) 一 一 一 


The answer using Bayes's theorem 


dit Search Mule Help 


P(x=3|lambda) 


The answer using Bayes's theorem 


Bayesian answer, alternate data set 


Xi = 114-7 45, 2.01 


бо 


Inferring parameters in science experiments 


A source emits particles; they decay at locations 
exponentially distributed with lengthscale A. De- 
can be observed only if the location falls in a 
window (Zmin, Zmax) = (a,b) = (1,20). 


N decays are observed at locations (21,...,: EN). 
› 


What is A? 


+ What about the possibility that the distribution is a mixture 
of two exponentials? 
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maximum likelihood distribution 
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maximum likelihood distribution 
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P(Data | lambda1, lambda2, k1..kN = 00000111) 
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P(Data | lambda1, lambda2, k1..kN = 11111111) 
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Marginal likelihood of lambda1 and lambda2 - P(Data | lambda1,lambda2) 


1 
og(lambda2) 


2 
одате 


P(k1..KN | Data) 
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Posterior distribution of the labels 


P(x | lambda1*, lambda2*) 
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ML model with two lambda's 
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Data set 3 - 8 points 


Data = 0.64 0.016 0.100 0.050 3.09 2.14 0.200 7.10 


Model 1 
optimized lambda = 1.6637 
ML тодей = 5.6254e-06 
Evidence1 = 5.4555e-07 

@Model 2 


Evidence2 = 1.3994e-05 
ML = 2.5026e-04 
tau1ml = 0.080025 : tau2ml = 2.8289 
k1..KN optimized (optimal labels) = 01110010 
EvidenceRatio = 25.651 
PosteriorProbOfH2 = 0.96248 
PosteriorProbOfH1 = 0.037522 


= До 63 дъ 


P(Data | lambda1, lambda2, k1..kN = 11000000) 


F(K1..KN | Data) 


Posterior distribution of the labels 


300 


Pix | lambdat”, lambda2”) 


Clustering 


K means clustering 


Clustering 


Recommended homework 


(Noisy channels - Chapters 8, 9, 10 (10.1-10.4 only) 
^ Exercises 9.17 (p155); 10.12 (172); 15.12 (235) 
© and (if you want more practice) 15.11, 15.13, 15.15 


@nvent a channel to pose to your colleagues: 
@'what's the capacity of this ?' 


The reading associated with the current lectures is Chapters 3, 21 
(especially sec 21.2), and 22 (especially sec 22.1), and 27. 


Other recommended exercises are listed on handout 2. 


www.inference.phy.cam.ac.uk/itprnn/ 


